1. Field
The present inventive concept pertains to a system and method to interpret and describe program behavior via instrumentation of virtualization events. The present inventive concept more particularly concerns a system and method to create a number of passive breakpoints in a virtual machine via instrumentation of common virtual machine trapping events.
2. Discussion of Related Art
Many virtualization platforms provide a process to analyze state and behavior of a virtual machine. Such processes are useful for a variety of applications including automated software analysis.
Static analysis of a virtual machine involves inspecting a snapshot of volatile memory as the virtual processor state is paused. By contrast, dynamic analysis involves observing changes to volatile memory and processor state in a running virtual machine to observe the presence of specific behaviors. In observing such specific behaviors, analysts may better understand objectives of malware that may be present within the virtual machine. Examples of dynamic analysis of a virtual machine may include tracking specific function calls or logging reads or writes to specific regions of memory as software executes.
Three conventional mechanisms or strategies for analyzing malware include hardware breakpoints, software breakpoints, and single stepping. Setting breakpoints is a useful mechanism for dynamic analysis, which provides the ability to instrument program execution for a particular piece of software. More specifically, the term “breakpoint” refers to a mechanism that triggers control of execution when software has attempted to execute a specific instruction or set of instructions at a specific location in memory.
When applied to virtualization, using breakpoints can be thought of as a means for controlling the execution of code by a virtual machine using control software running in an entirely separate virtual machine. Note that this is technically different than kernel debuggers, e.g., those commonly used with operating systems sold under the trademarks Linux® and Windows®, as no local debugging mode or software within the virtual machine is necessary. Setting breakpoints is in many cases similar to the concept of “inline function hooking.” Inline function hooking performs a modification of the software under analysis in which the first few instructions of a function are replaced in order redirect execution by analysis (or malicious) software, before returning the flow of control to the intended function. This method achieves the same effect without modifying instructions within the virtual machine.
Hardware breakpoints are used for the purposes of debugging; hardware virtualization platforms include debug registers intended to be used for this specific purpose. Modern architectures, however, contain a limited number, e.g., 4 on x86, and at most 6 on current implementations of ARM. These limited numbers are inadequate for effective malware analysis in many situations, such as when the type of analysis being employed potentially requires hundreds or thousands of code instrumentation hooks to observe software behavior.
Traditional software breakpoints involve directly modifying the code within the guest virtual machine. For example, setting a breakpoint on the x86 architecture involves writing the opcode O×CC or “INT3” instruction to memory at the location of the breakpoint (also called the instrumentation point). Within virtual machine platforms, it is possible to use software breakpoints within a virtual machine. This method is not passive because it requires direct modification of software, which may result in modified software behavior; malware is often structured to protect against such modification. Examples of how malware protects against such modification include actively scanning for software breakpoints or by calculating and verifying check sums on code areas, or measuring code by some other means, to determine if the code has been modified before execution. The presence of software breakpoints in these circumstances will thus alter program execution and prevent accurate determination of intended program behavior.
Another option for code analysis within hardware assisted virtual machine platforms is to single step one instruction at a time within a virtual machine while monitoring a state between each instruction. In practice, for any software running within a modern operating system, this method is far too slow and inevitably causes instabilities within the virtual machine operating system.
Conventional mechanisms for dynamic analysis of software in virtual machine environments are not ideal for analysis of malware. Malware often abuses normal software conventions with respect to operating systems as well as system hardware architecture, which confounds traditional analysis. Conventional analysis can be slow, inadequate, and/or inaccurate. Thus, there is a demand for a system and method operable to describe program behavior via instrumentation of virtualization events that does not suffer from the aforementioned deficiencies.